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-- The MAILING DATE of this communication appears on the cover sheet with the correspondence address ~ 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 



- Extensions of time maybe available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

Responsive to communication(s) filed on 18 May 2005 . 
2a)D This action is FINAL. 2b)l3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) S Claim(s) 1-9 and 11-26 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) S Claim(s) 1-9, 11-26 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) K The drawing(s) filed on 26 April 2001 is/are: a)l3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

11) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) [3 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)SAII b)D Some*c)n None of: 

1 .13 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 1 9(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 

Attachment(s) 

1 ) E3 Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No(s). . 

2) D Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-152) 

3) Q Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) O Other: 
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DETAILED ACTION 

1. A request for continued examination under 37 CFR 1. 1 14, including the fee set 
forth in 37 CFR 1.1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1. 1 14, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1. 1 14. Applicant's submission filed on 19 April 
2005 has been entered. Applicants have now cancelled claim 10. Claims 1-9, and 11- 
26 are currently pending in this application. Claims 1-9, and 11-26 remain rejected 
based on new grounds for rejection. 



Response to Arguments 

2. Applicants arguments filed on 19 April and 1 8 May of 2005 have been fully 
considered. 

Regarding applicants response to 112(1) rejections : The examiner withdraws the 
1 12(1) rejection in view of applicants arguments filed 19 April and 18 May of 2005. In 
summary, the rejection is withdrawn in view of applicants arguments that the claimed 
features relating to "comparing the information data of the installation components with 
picture data", "identifying components in the picture data", and " deriving hypotheses 
regarding the identified components", are realized using conventional techniques which 
are well known in the art of image recognition and processing (page 15, paragraph 1, 
page 16, paragraphs 1 & 3). 
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Regarding applicant's response to 103(a) rejection : Applicant's arguments with 
respect to the previous 103(a) rejection of claims 7-9, and 11-26 have been considered 
but are moot in view of the new ground(s) of rejection. (Please see new art rejection 
below) 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 ILS.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 

USPQ 459 (1966), that are applied for establishing a background for determining 

obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

3. Claims 1-9, and 11-26 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over "Three Dimensional Object Recognition", Besl et al, 

Computing Surveys, Vol. 17, No. 1, March 1985 in view of U.S. Patent 

5,740,341 issued to Oota et al or "Reconstruction of 3D Virtual Buildings fro 2D 

Architectural Floor Plans", So etal, VRST98', ACM 1-58113-019/98/0011, ACM 

1998. 
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Independent claim 1 is drawn to: 

Generating an image of installation model by: 
Memory (1 st ) for storing picture data 
Memory (2 nd ) for storing component information 
Memory (3 rd ) for storing virtual installation model 

Evaluation and control unit for comparing component information data with real picture data 
Identifying components in picture data as installation components 
Deriving a hypotheses for identified components in picture data 
Generating respective installation components in virtual installation model 

Regarding independent claim 1 : Besl discloses computer based methods and 
systems for the identification of 3D objects (components) from 2D image (picture) data 
(Sections 2.0-4.0) inclusive of comparing object information data with image (picture) 
data and identifying the objects. (Sections 5.2-5.6, page 124, col. 2, line 9, Figs. 39, 
40) That is, Besl renders obvious the claimed features of the evaluation and control 
unit by setting forth a functionally equivalent process. (See Figs. 29, 39, 40, especially 
Fig. 29) In particular, Besl discloses object recognition programs such as IMAGINE 
(page 119, paragraph 3-4), which identify objects by evaluating primitives (geometric) 
representing a physical object (component) where the image is specified as a set of 
parameters that correspond to locations in the image model. (Sections 5.3-5.6, also 
see ACRONYM, Section 5.8) The recognition process includes the use of an object 
model library (page 122-124) and the process of deriving a hypothesis for identified 
objects (page 1 19, column 2, page 130, column 2) in the image. Obviously, the system 
disclosed in Fig. 29, page 110 of Besl includes a memory for storing the digitized 
picture and object information, otherwise the device would not operate. 

Besl does not explicitly teach displaying a virtual representation of the facilities 
installation of components. 
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Oota and So both teach systems capable of displaying and modeling a virtual 
representation of installed components (i.e. a virtual model) in a facility or building. 
Specifically, Oota teaches system for displaying a model of a virtual building including 
components (piping, etc.) that have been interactively arranged to form a virtual model. 
(Abstract CL2-L3-41, CL16-L6-11, Figs. 3, 10-12) So teaches the use of commercially 
available virtual reality modeling software (3D Studio Max, Kinetex 3D, Softimage 3D, 
AutoCAD, etc.) in creating a virtual installation model of a building that includes 
installation components. (Abstract, Sections 1,2.0, 2.2, 3.2, Figs. 1, 4, 5, 8, 10-16) 

It would have been obvious to one having ordinary skill in the art at the time the 
claimed invention was made to modify the teachings ofBesI relating to the identification 
of 3D objects (components) from 2D image (picture) data, with the teachings of Oota or 
So relating to the virtual modeling of components in a building, to realize the elements 
of the claimed invention. An obvious motivation exists since, turning to the prior art, we 
find successful use of picture (image) data in the virtual modeling of buildings. (See: 
Marks, CL3-L2-9, Fig. 2) Further, the level of skill required by an artisan to realize the 
claimed limitations of the present invention is clearly established by all references. (See: 
Besl/Oota or So, Background/Abstract) Accordingly, a skilled artisan tasked with 
developing a system requiring object (component) recognition from image data 
(pictures) and virtual modeling of building component, and having access to the 
teachings ofBesI and Oota or So, would have known to turn to the prior art, and 
knowingly modified the teachings ofBesI, with the teachings of Oota or So (or visa 
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versa), to realize the claimed elements of the present invention while gaining the 
advantages of reduced cost and development time. 

Regarding dependent claims 2-4 : Besl teaches performing image analysis on 
picture data and an object library as previously cited above. Oota and So both teach 
using virtual components (installation components) from a library of components 
representing the total virtual system (Oota: CL5-L15-59, Fig. 2 (component/part 
database, So: Section 2.0). Oota and So further teach objects being located by 
geometric information and multiple window views of picture data and 3D (virtual) model 
views (Oota: Figs. 11(1-3), 15, 17, So: Figs. 3, 4, 10-12, 1 4-1 6) and would have 
knowingly been incorporated by a skilled artisan using the reasoning previously cited 
above. 

Regarding dependent claims 5-9 : Oota and So disclose building a 3D (virtual) 
model using the well-known techniques of "dropping", "clicking and dragging", and 
"rubber banding" in the manipulation of primitives based on geometric information 
(Oota: CL2-L38, So: Section 2.0, Fig. 1). These features are also obvious OS features 
(such as Windows), and would have knowingly been incorporated by a skilled artisan. 
(See: Microsoft Computer Dictionary, 3 rd Edition 1997, terms: drop, drag, rubber 
banding) Oota and So also teach the evaluation of structural components (by function) 
to assign primitives (add components) in a 3D (virtual) installation model. (Oota: CL5- 
L15-59, Fig. 2, So: 3.2, 3.4, Fig. 1) 

Regarding dependent claims 11-12 : Besl teaches that the image data can be 
from any digital image source as noted above (see Fig. 29). Oota and So teach virtual 
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models from CAD system with memory and a multiple perspective view display as 
noted above (Oota: Figs. 11(1-3), 15, 17, So: Figs. 3, 4, 10-12, 14-16) 

Independent claim 13 is drawn to: 

Generating an image of real installation model by: 
Generating picture data 

comparing component information data with picture data 
Identifying components in picture data as installation components 

Regarding independent claims 13 and 25 : As previously cited, Best discloses 
computer based methods and systems for the identification of 3D objects 
(components) from 2D image (picture) data (Sections 2.0-4.0) inclusive of comparing 
object information data with image (picture) data and identifying the objects. (Sections 
5.2-5.6, page 124, col. 2, line 9, Figs. 39, 40) That is, Besl renders obvious the claimed 
features of the evaluation and control unit by setting forth a functionally equivalent 
process. (See Figs. 29, 39, 40, especially Fig. 29) In particular, Besl discloses object 
recognition programs such as IMAGINE (page 11 9, paragraph 3-4), which identify 
objects by evaluating primitives (geometric) representing a physical object (component) 
where the image is specified as a set of parameters that correspond to locations in the 
image model. (Sections 5.3-5.6, also see ACRONYM, Section 5.8) The recognition 
process includes the use of an object model library (page 122-124) and the process of 
deriving a hypothesis for identified objects (page 119, column 2, page 130, column 2) 
in the image. Obviously, the system disclosed in Fig. 29, page 110 of Besl includes a 
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memory for storing the digitized picture and object information, otherwise the device 
would not operate. 

Best does not explicitly teach displaying a virtual representation of the facilities 
installation of components. 

Oota and So both teach systems capable of displaying and modeling a virtual 
representation of installed components (i.e. a virtual model) in a facility or building. 
Specifically, Oota teaches system for displaying a model of a virtual building including 
components (piping, etc.) that have been interactively arranged to form a virtual model. 
(Abstract CL2-L3-41, CL16-L6-1 1, Figs. 3, 10-12) So teaches the use of commercially 
available virtual reality modeling software (3D Studio Max, Kinetex 3D, Softimage 3D, 
AutoCAD, etc.) in creating a virtual installation model of a building that includes 
installation components. (Abstract, Sections 1, 2.0, 2.2, 3.2, Figs. 1, 4, 5, 8, 10-16) 

It would have been obvious to one having ordinary skill in the art at the time the 
claimed invention was made to modify the teachings ofBesI relating to the identification 
of 3D objects (components) from 2D image (picture) data, with the teachings of Oota or 
So relating to the virtual modeling of components in a building, to realize the elements 
of the claimed invention. An obvious motivation exists since, turning to the prior art, we 
find successful use of picture (image) data in the virtual modeling of buildings. (See: 
Marks, CL3-L2-9, Fig. 2) Further, the level of skill required by an artisan to realize the 
claimed limitations of the present invention is clearly established by all references. (See: 
Besl/Oota or So, Background/Abstract) Accordingly, a skilled artisan tasked with 
developing a system requiring object (component) recognition from image data 
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(pictures) and virtual modeling of building components, and having access to the 
teachings ofBesI and Oota or So, would have known to turn to the prior art, and 
knowingly modified the teachings ofBesI, with the teachings of Oota or So (or visa 
versa), to realize the claimed elements of the present invention while gaining the 
advantages of reduced cost and development time. 

Regarding dependent claims 14-16 : Besl teaches performing image analysis on 
picture data and an object library as previously cited above. Oota and So both teach 
using virtual components (installation components) from a library of components 
representing the total virtual system (Oota: CL5-L15-59, Fig. 2 (component/part 
database, So: Section 2.0). Oota and So further teach objects being located by 
geometric information and multiple window views of picture data and 3D (virtual) model 
views (Oota: Figs. 11(1-3), 15, 17, So: Figs. 3, 4, 10-12, 14-16) and would have 
knowingly been incorporated by a skilled artisan using the reasoning previously cited 
above. 

Regarding dependent claims 17-24 : Oota and So disclose building a 3D (virtual) 
model using the well-known techniques of "dropping", "clicking and dragging", and 
"rubber banding" in the manipulation of primitives based on geometric information 
(Oota: CL2-L38, So: Section 2.0, Fig. 1). These features are also obvious OS features 
(such as Windows), and would have knowingly been incorporated by a skilled artisan. 
(See: Microsoft Computer Dictionary, 3 rd Edition 1997, terms: drop, drag, rubber 
banding) Oota and So also teach the evaluation of structural components (by function) 
to assign primitives (add components) in a 3D (virtual) installation model. (Oota: CL5- 
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L15-59, Fig. 2, So: 3.2, 3.4, Fig. 1) Both Oota and So teach an automatic process 
(Oota: Abstract, So: Sections 2-3) 

Per dependent claim 26 : Besl teaches that the image data can be from any 
digital image source as noted above (see Fig. 29). Oota and So teach virtual models 
from CAD system with memory and a multiple perspective view display as noted above 
(Oota: Figs. 11(1-3), 15, 17, So: Figs. 3, 4, 10-12, 14-16) 

Conclusion 

4. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure, careful consideration should be given prior to applicant's 
response to this Office Action. 

U.S. Patent 5,887,041 issued Zachar et al teaches component recognition from picture 
data. 

U.S. Patent 6,404,913 issued to Ohki teaches synthesized graphics from photographs. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Fred Ferris whose telephone number is 571-272-3778 
and whose normal working hours are 8:30am to 5:00pm Monday to Friday. Any inquiry 
of a general nature relating to the status of this application should be directed to the 
group receptionist whose telephone number is 571-272-3700. If attempts to reach the 
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examiner by telephone are unsuccessful, the examiner's supervisor, Jean Homere can 
be reached at 571-272-3780. The Official Fax Number is: (703) 872-9306 

TredTerris, Patent Examiner 
Simulation and Emulation, Art Unit 2128 
U.S. Patent and Trademark Office 
Randolph Building, Room 5D19 



Alexandria, VA 22313 
Phone: (571-272-3778) 
Fred.Ferris@uspto.gov 
July 8, 2005 



401 Dulany Street 




